Visual axis detection method

ABSTRACT

An accumulation-type image pickup device having first and second accumulation periods receives light relfected from an eye illuminated by a light beam from an illumination device. A memory for storing a signal from the image pickup device stores an image signal of the eye generated in one of the two accumulation periods of the image pickup device. The illumination device emits light in only one of the two accumulation periods. A differential signal generation circuit generates a differential signal between an image signal generated by the image pickup device during the first accumulation period and an image signal generated in the second accumulation period. The visual axis of the eye is determined on the basis of the signal from the differential signal generation circuit.

This application is a continuation of application Ser. No. 08/357.784,filed Dec. 16, 1994, now abandoned, which is a continuation ofapplication Ser. No. 08,139,806 filed Oct. 22, 1993, now abandoned.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a visual axis detection apparatus, andespecially to a visual axis detection apparatus which detects an axis inan observation point direction of a viewer (photographer) or a so-calledvisual axis when the viewer observes an observation plane (imagingplate) on which an object image is formed by a photographing system inan optical system such as a camera, by utilizing a reflected image(eyeball image) formed when an eyeball of the viewer is illuminated withan infrared ray.

2. Related Background Art

Various visual axis so as detection apparatuses for detecting the visualaxis to detect a position on a view plane which the viewer (examinedperson) views have been proposed.

For example, in Japanese Laid-Open Patent Application No. 2-264632, aninfrared light beam from a light source is projected to an anterior partof the eye in an eye to be examined and an axis of vision (observationpoint) is determined by utilizing a cornea reflected image on the basisof light reflected from a cornea and a focus-imaging point on a pupil.

In a camera disclosed in Japanese Laid-Open Patent Application No.61-61135, a the direction of metering by a focus detection apparatus ismechanically controlled on the basis of an output signal from a visualaxis detection means to adjust a focal point state of a photographingsystem.

FIG. 5 is a schematic view of a visual axis detection apparatus proposedin Japanese Laid-Open Patent Application No. 2-264632, FIG. 6 is a graphof an output signal from one line of an image sensor of FIG. 5, and FIG.7 is a perspective view of a portion of a finder system when the visualaxis detection apparatus of FIG. 5 is applied to a single eye reflexcamera.

Numeral 101 denotes an eyeball of an examined person (observer), numeral1 denotes a cornea of the eyeball of the examined person, numeral 2denotes a sclera, and numeral 3 denotes an iris. O' denotes a center ofrotation of the eyeball 101, O denotes a center of curvature of thecornea 1, a and b denote ends of the iris 3, and e and f denotepositions where cornea reflected images are formed owing to lightsources 4a and 4b to be described hereinafter. Numeral 4a and 4b denotelight sources which may be light emitting diodes or the like foremitting infrared rays which are unpleasant for the examined person. Thelight source 4a (4b) is arranged closer to a projection lens 6a (6b)than to a focal plane of the projection lens 6a (6b). The projectionlenses 6a and 6b are applied for widely illuminating the cornea 1defining a light beam from the light sources 4a and 4b as diverged lightbeam.

The light source 4a lies on an optical axis of the projection lens 6aand the light source 4b lies on an optical axis of the projection lens6b, and they are arranged symmetrically along a z-axis direction withrespect to an optical axis aX₁.

Numeral 7 denotes a light receiving lens which forms the corneareflected images e and f formed near the cornea 1 and the ends a and bof the iris 3 on an image sensor plane 9. Numeral 10 denotes anarithmetic means which calculates the visual axis of the examined eye byusing the output signal from the image sensor 9. aX₁ denotes an opticalaxis of the light receiving lens 7 and it matches the X axis in FIG. 5.aX₂ denotes an optical axis of the eyeball which makes an angle θ withrespect to the X axis.

In this example, the infrared ray emitted from the light source 4a (4b)passes through the projection lens 6a (6b) and thereafter widelyilluminates the cornea 1 of the eyeball 101 in a diverging state. Theinfrared ray which passes through the cornea 1 illuminates the iris 3.

The cornea reflected images e and f based on the light beam reflected bythe surface of the cornea 1 of the infrared rays for illuminating theeyeball are reformed at points e' and f' on the image sensor 9 throughthe light receiving lens 7. In FIGS. 5 and 6, e' and f' denoteprojection images of the cornea reflected image (virtual images) e and fformed by a set of light sources 4a and 4b. The centers of theprojection images e' and f' substantially match to the projection pointon the image sensor 9 of the cornea reflected image formed when theillumination means is arranged on the optical axis aX₁.

The infrared ray which is diffusion-reflected by the surface of the iris3 is directed to the image sensor 9 through the light receiving lens 7to form the iris image.

On the other hand, the infrared ray transmitted through the pupil of theeyeball to illuminate a retina has the wavelength of the infrared rangeand the illuminated area is an area of a low view cell density which isapart from a center area, so that the examined person cannot recognizethe light sources 4a and 4b.

The ordinate in FIG. 6 represents an output I along the z-axis directionof the image sensor 9. Since most of the infrared rays transmittedthrough the pupil are not reflected back, there is no difference in theoutput at the boundary between the pupil and the iris 3. As a result,the iris images a' and b' at the ends of the iris can be detected.

When an area sensor having a two-dimensional photo-sensor array is usedas the image sensor 9 of FIG. 6, two-dimensional light distributioninformation of the reflected image (eyeball image) is obtained from thefront eye as shown in FIG. 8.

In FIG. 8, numeral 141 denotes a light receiving area of the imagesensors, E' and F' denote cornea reflected images of the light sources4a and. 4b, A' denotes a boundary between the iris and the pupil, and G'denotes a boundary between the sclera 2 and the cornea 1. Since thereflectivities of the sclera 1 and the iris 3 are not substantiallydifferent from each other in the infrared range, the boundary G' can notbe clearly discriminated by a naked eye. J' denotes an image of a lowereyelid, K' denotes an image of an upper eyelid and L' denotes an imageof eyelashes.

In order to detect the direction of the visual axis from the eyeballimage of the front part of the eye, it has been known to calculate therelative relation between the cornea reflected images E' and F' (or anintermediate image of E' and F') and the position of the center ofpupil. Various methods for determining the center of pupil have beenknown. For example, an output of one particular line of the image sensoris sampled to calculate a center point of the pupil edge positions a'and b' of FIG. 6. Alternatively, the output information of the areasensor may be used to sample a number of pupil edge points andthereafter determine the center point by a least square approximation.

Optional equipment having a finder system such as a still camera or avideo camera is frequently used in out-of-door conditions. When suchoptical equipment is used out-of-doors, the eyeball of the photographeris illuminated by an external ray. Thus, an image forming light beamreceived by the image sensor 9 includes not only the image of the frontpart of the eye illuminated with the light sources 4a and 4b but also acomplex image affected by the disturbance by the external ray.

The external ray causing the most problems is a direct light incident onthe front part of the eye from the sun. The energy of sunlight is verystrong and includes a large amount same spectrum components as thoseemitted by the light sources 4a and 4b. Accordingly, it is difficult tofully eliminate the external ray by spectrum means such as a visible raycut filter.

When the front part of the eye is illuminated by sunlight, a variety ofdisturbances are generated in the image and the external ray componentis stronger than the infrared component. As a result, a pattern (eyeballimage) cannot be substantially discriminated. When the external rayexists, the brightness in the pupil which should be at a lowestbrightness level of luminescence (between a' and b' in FIG. 6) becomeshigher or declines so that the detection of the pupil edges and hencethe position of the center of the pupil cannot be correctly determined.

When the neighborhood of the boundary of the sclera and the iris isstrongly illuminated, an obscure edge, which inherently seems unclear,rises to the surface or declines therein, so that the pupil edges aremisdetected. When the eyelashes grow downward, they are illuminated bythe external ray, so that they may be misdetected as the pupil edge.Since the eyelashes extend out of the face in contrast to the eyeball,they are easily subject to the illumination by the external ray.

Such a misdetection occurs not only for the pupil edge but also for thecornea reflected images e and f of the light sources 4a and 4b. When theends of the eyelashes are directly illuminated by the sunlight, theybecome strong brilliant points, which are misdetected as the corneareflected images. When eyeglasses are put, on dust deposited on theeyeglasses may be highlighted.

Besides sunlight, a downwardly directed light having high luminescenceand various artificial light sources may form also an external e ray.When eyeglasses are put, on a distance between the eyepiece portion inthe finder system and the eyeball generally becomes separated so thatthe external ray easily enters into the eye. Further, the reflectioncoming from the lens surfaces of the eyeglasses is adversely affected.

When the visual axis is to be detected by using the image signal fromthe image sensor, an accumulation-type image sensor is frequently usedin view of the sensitivity requirements for the system. As a result,there has been a problem that DC noise elimination by an AC coupling ora period detection system which is usually used in a single sensor cellcannot be used.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a visual axisdetection apparatus for detecting an eyeball image by usingaccumulation-type image pickup means which reduces the affect of noisedue to an external ray and detects the visual axis of the eyeball of thephotographer (the examined person) who looks into a finder, by properlysetting an accumulation method of the eyeball image (image information)by the image pickup means and a processing method of the imageinformation based on the eyeball image from the image pickup means.

In the visual axis detection apparatus of the present invention, theeyeball of the examined person is illuminated by a light beam comingfrom illumination means, an eyeball image based on a reflected lightfrom the eyeball is formed on a surface in accumulation-type imagepickup means, an image signal from the image pickup means is stored inmemory means, and a visual axis of the eye of the examined person iscalculated by utilizing the image signal stored in the memory means. Theimage pickup means has first and second accumulation periods and thememory means stores the image signal of the eyeball generated in one ofthe two accumulation periods and the illumination means emits a light inone of the two accumulation periods. A difference signal between theimage signal from the image pickup means generated in the firstaccumulation period and the image signal generated in the secondaccumulation period is determined by differential signal generationmeans and the visual axis of the examined person is detected based onthe signal from the differential signal generation means.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a main schematic view of the first embodiment;

FIG. 2 shows a flowchart of the first embodiment;

FIG. 3 shows a main schematic view of an image sensor of a secondembodiment;

FIG. 4 shows a flowchart of the second embodiment;

FIG. 5 shows a main schematic view of a conventional visual axisdetection apparatus;

FIG. 6 shows a graph of the output signal from the image sensor in FIG.5;

FIG. 7 shows a main schematic view when the visual axis detectionapparatus is applied to a single reflex camera;

FIG. 8 shows an eyeball image formed on an area sensor;

FIG. 9 shows a view when the visual detection apparatus is mounted intoa single reflex camera; and

FIG. 10 shows a block diagram for explaining how the apparatus in FIG. 9is controlled.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 shows a schematic diagram of the first Embodiment of the presentinvention and FIG. 2 shows a flow chart for explaining the visual axisdetection in the first Embodiment.

In the present embodiment, in contrast to the conventional visualdetection apparatus of FIG. 5, a photo-electrically converted signalfrom the image sensor 9 which functions as the accumulation-type imagepickup means is processed by arithmetic means 101, and a RAM (memory) 21for storing the data from the arithmetic means 101 is further provided.Specifically, the visual axis operation method is improved in thearithmetic means 101 by using the data stored in the RAM 21 to eliminatethe adverse affects of an external ray.

The elements of the present embodiment are now explained in sequencealthough it may be partially duplicate the description for FIG. 5.

In FIG. 1, numeral 45 denotes an eyeball of an examined one (viewer),numeral 1 denotes a cornea of the eyeball of the examined one, numeral 2denotes a sclera and numeral 3 denotes an iris. O' denotes a center ofrotation of the eyeball 101, O denotes a center of curvature of thecornea 1, a and b denote ends of the iris 3, and e and f denotepositions where cornea reflected images are generated by light sources4a and 4b to be described hereinafter. Numerals 4a and 4b denote lightsources which may be light emitting diodes for emitting infrared rayswhich are unpleasant to the examined one. The light source 4a (4b) isarranged closer to a projection lens 6a (6b) than to a focal plane ofthe projection lens 6a (6b). The projection lenses 6a and 6b convert thelight beams from the light sources 4a and 4b to diverging lights towidely illuminate on a surface of the cornea 1.

The light source 4a lies on an optical axis of the projection lens 6awhile the light source 4b lies on an optical axis of the projection lens6b and they are arranged symmetrically along a z-axis relative to anoptical axis aX₁. The light sources 4a and 4b and the projection lenses6a and 6b form the illumination means.

Numeral 7 denotes a light receiving lens which focuses the corneareflected images e and f formed in the vicinity of the cornea 1 and theends a and b of the iris 3 onto the image sensor 9. The light receivinglens 7 and the image sensor 9 form one of the light receiving meanswhich converts the light from the eye into an electrical signal.

Numeral 101 denotes an arithmetic means which calculates the visual axisof the examined person by using the output signal from the image sensor9, as will be described hereinafter. The basic detection method thereforis described in Japanese Laid-Open Patent Application No. 4-447127.Numeral 11 denotes a RAM which functions as memory means which storesdata calculated by the arithmetic means 101. aX₁ denotes an optical axisof the light receiving lens 7, which matches an X-axis of a graph of theoutput of the sensor 9. aX₂ denotes an optical axis of the eyeball whichmakes angle θ with respect to the X axis.

In the present embodiment, the infrared ray emitted from the lightsource 4a (4b) passes through the projection lens 6a (6b) and thereafterdiverges to widely illuminate the cornea 1 of the eyeball 45. Theinfrared ray transmitted through the cornea 1 illuminates the iris 3.

The cornea reflected images e and f based on the light beam reflected bythe surface of the cornea 1, of the infrared rays illuminating theeyeball are reimaged onto the points e' and f' on the image sensor 9through the light receiving lens 7. In FIGS. 1 and 6, e' and f' denoteprojection images of the cornea reflected images (virtual images) e andf generated by the set of light sources 4a and 4b. A mid-point of theprojection images e' and f' substantially matches the projectionposition of the cornea reflected image on the image sensor 9, which isgenerated when the illumination means is arranged on the optical axisaX₂.

The infrared ray which is diffusion-reflected by the surface of the iris3 is introduced into the image sensor 9 through the light receiving lens7 to form the iris image.

On the other hand, the infrared ray transmitted through the pupil of theeyeball illuminates the retina and is absorbed thereby. However sincethe illuminated area has a low density of viewing cells which is apartfrom the center, the examined one cannot discriminate the light sources4a and 4b.

In FIG. 6, an ordinate represents an output I in the z-axis of the imagesensor 9. Since most of the infrared rays transmitted through the pupilare not reflected back, there arises a difference in the outputs at theboundary between the pupil and the iris 3 and the iris images a' and b'of the iris edges are detected.

In the present embodiment, the arithmetic means 101 respectively detectscoordinates (Za', Zb' and Ze', Zf') of peculiar points (a', b' and e',f') on the eyeball on the image sensor 9 based on a flow chart of FIG.2, and calculates a rotation angle θ of the eyeball in accordance with aformula:

    β·OC·sin θ≅(Za'+Zb')/2-(Ze'+Zf')/2

where β is a magnification factor of the light receiving optical system(≅L₀ /L₁).

A vision angle of the eyeball is determined from the rotation angle θ todetermine the subject viewed by the examined one.

In the line of vision detector of the present invention, a distance L₁between the position at which the cornea reflected image is generatedand the light receiving lens 7 satisfies the relation:

    (L.sub.1 |Ze'-Zf'|)/L.sub.0 Z.sub.0 ≅OC/(L.sub.1 -L.sub.2 +OC)

where Z₀ is a spacing in the z-direction of the set of light sources 4a(4b), and L₂ is a spacing in the x direction between the light source 4a(4b) and the light receiving lens 7.

Thus, even if the distance between the line of vision detector and theeyeball changes, the distance L₁ may be calculated from the spacing|Ze'-Zf'| of the two cornea reflected images.

The operation of the visual axis detection apparatus is now explainedwith reference to the flow chart of FIG. 2.

In a step 201, the detection operation of the visual axis starts. In astep 202, the light sources 4a and 4b are turned on and at substantiallythe same time, the process proceeds to a step 203 to start the firstaccumulation operation of the image sensor 9. The accumulation by theimage sensor 9 may be controlled by comparing a real time accumulationamount motor signal with a predetermined reference, or by time controlby a hardware or software timer.

The process proceeds to a step 204 at substantially the end of the firstaccumulation time of the image sensor to turn off the light sources 4aand 4b. The photo-electric conversion signals of the image sensor 9 aresequentially read through a loop of steps 205-207 and the A/D convertedelectrical signals Pi of the cells are stored in the memory (RAM) 21.Where the image sensor 9 itself does not have a memory function, theimage sensor 9 may sense the light and error-move during reading thesignals. Accordingly, the loop is designed to be completed in asufficiently short time in comparison with the accumulation time.

Where the image sensor 9 includes an analog memory function, the signalcharges may be temporarily shifted to the non-photosensitive memory andsequentially read into digital system at a low speed. The memoryfunction of the present embodiment may be implemented as a CCD channelor a capacitor array.

When the reading and storing of all of the required pixels arecompleted, the process proceeds to a step 208 to conduct the secondaccumulation operation. The accumulation time of the second accumulationoperation is substantially the same as the accumulation time of thefirst accumulation performed in the step 203.

In the second accumulation operation, the light sources 4a and 4b arenot turned on and the front eye image is sampled by only the externalray illumination to cancel the external ray components. In the presentembodiment, the accumulation time may be reduced to one half and theread gain may be doubled in order to reduce the accumulation time whilekeeping the apparent signal quantity.

When the second accumulation operation is finished, the photoelectricconversion signals of the image sensor are sequentially read through aloop of steps 209-211.

Then the arithmetic means 101 reads the signals Pi of the same pixelsproduced in the first accumulation, calculates differences di betweenthe signals Pi and the current signals Pi' and restores the result inthe memory 21.

In the present embodiment, the arithmetic means 101 also includesdifferential signal generation means for determining the differentialsignal Pi'. This operation is carried out for all the pixels so that thememory 21 has an image signal based on the eyeball image whichsubstantially eliminates the contribution of the external ray due to thesunlight or the like. In the present embodiment, the direction of thevisual axis is calculated in a step 212 based on the above image signalso that any malfunction is prevented and the highly accurate detectionof the visual axis is attained.

FIG. 3 shows a schematic view of an image sensor (a sensor chip) 301 ina second Embodiment of the present invention, and FIG. 4 shows a flowchart of the operation of the present embodiment. Other elements of thepresent embodiments are substantially identical to those of the firstEmbodiment.

In FIG. 3, numeral 301 denotes a sensor chip having a well-knownself-scanning system and a power supply and the like, and it is shown asa functional block in FIG. 3 for simplification.

The sensor chip 301 is provided at the position of the image sensor 9 inFIG. 1. Numeral 302 denotes a photo-sensing block which is a CCD sensorhaving M×N areas. FIG. 3 shows a frame-transfer-type system which sharesthe photo-sensing unit with a transfer unit although the same functionmay be attained by an interline-type system. A masked column 303 isprovided at a left end of the photo-sensitive area. It is a monitorpixel to detect a dark signal level. A transfer buffer 311, a firstmemory 321, a transfer buffer/horizontal read register 331, a secondmemory unit 341, a transfer buffer/horizontal read register 351, and adifferential amplifier 361 are provided in sequence. The elements otherthan the photo-sensitive area are fully shielded from the light by analuminum film or the like.

The operation of the present embodiment is now explained with referenceto a flow chart of FIG. 4.

In a step 401, the visual axis detection operation starts. In a step402, the light sources 4a and 4b are turned on. At substantially sametime, the first accumulation operation of the image sensor 301 isstarted in a step 403, and after employing the accumulation to apredetermined monitor level or after a predetermined time, theaccumulation is terminated.

In a step 404, the light sources 4a and 4b are turned off, and in a step405, the transfer operation is conducted. In the transfer operation, thesignal charges accumulated in the photo-sensing unit 302 of the imagesensor 301 are transferred to the memory unit 321 through the transferbuffer 311.

The transfer method is well-known. In the illustratedframe-transfer-type system, the signal charges of the pixels aretransferred downward one line per one clock signal. The entire image istransferred to the memory unit 321 by (N+1) clock signals includingthose for the buffer. It is necessary that the time required for thetransfer is sufficiently shorter than the accumulation time. In thepresent embodiment, the transfer rate of the CCD channel is determinedby the hardware and it is sufficiently high, so that any problem doesnot arise.

When the transfer is over, the process proceeds to a step 406 to conductthe second accumulation operation. Since the charges of thephoto-sensing unit are evacuated by the previous transfer operation, areset operation is not necessary but it may be conducted prior to thesecond accumulation if the circuit is designed to conduct the resetoperation.

In the second accumulation, the light sources 4a and 4b are not turnedon and the signal charges by only the external ray are accumulated.After completing the accumulation, the process proceeds to a step 407 toconduct the transfer.

In the transfer operation, the signal charges accumulated in thephoto-sensing unit 302 are transferred to the memory unit 321 and at thesame time the signal charges stored in the memory 321 by the firstaccumulation are transferred to another memory unit 341. Since they areprocessed simultaneously and in parallel the signal charges of the twoaccumulations are not mixed and the transfers are completed by (N+1)clock signals. Finally, the signal charges by the first accumulation arestored in the memory 341 and the signal charges of the secondaccumulation are stored in the memory 321.

In the next sequence, the signal is read outwardly through a loop ofsteps 408-409. This sequence may be at a lower speed than the transferin the sensor chip owing to an external radial circuit but the signalmay be read without regard to the sensing by the sensor because thesignal charges have been transferred to the light-shielded memory unit.

The signal charges stored in the memory units (321 and 341) aresequentially transferred, pixel by pixel, to the charge-voltageconverters 332 and 352 by the function of the horizontal line readregisters 331 and 351, and the signal voltages are applied to the inputterminals of the differential amplifier 361. Since both horizontalregisters 331 and 351 are operated by one clock simultaneously, thesignals of the same pixel of the photo-sensing unit 302 produced in thefirst and second accumulations are simultaneously applied to thepositive and negative inputs of the differential amplifier 361. As aresult, the image signal which the external ray components is subtractedtherefrom appears at the output terminal 371. When it is done for allpixels, the process proceeds to a step 410 to calculate the visual axis.In the present embodiment, the problems caused by the external ray iseliminated in such a manner as to attain a highly reliable signal.

In the present embodiment, a capacitor array may be also used toeliminate the external ray in the sensor chip. An image sensor whichtemporarily stores the photo-excited image signal charges in thecapacitor array through a current amplifier element such as a transistorand thereafter sequentially read them out has been known, and hence theelimination of the external ray which is functionally equivalent to theCCD arrangement described above may be attained.

Only one set of the memory unit may be provided for the firstaccumulation signal and the second accumulation signal may be subtractedon the chip and the result is output. Alternatively, it may be re-storedin the memory. The significance of the present invention is not limitedby the specific details of the implementation.

FIG. 9 shows a schematic diagram of an embodiment in which the line ofvision detector of the present invention is applied to a single reflexcamera.

In FIG. 9, numeral 31 denotes a photographing lens which comprises twolenses for the sake of convenience although it actually comprises morelenses. Numeral 32 denotes a main mirror which is skewed into aphotographing path or retracted therefrom depending on a view state ofan object by a finder system and a photographing state of an objectimage. Numeral 33 denotes a sub-mirror which reflects a light beamtransmitted through the main mirror 32 to a focal point detectionapparatus 39 at a bottom of a camera body to be described later.

Numeral 34 denotes a shutter and numeral 35 denotes a photo-sensingmember such as a silver salt film, CCD or MOS or the like solid stateimage pickup device, or an image pickup tube such as a videcon.

Numeral 36 denotes a focal point detection apparatus which comprises afield lens 36a arranged near a focusing plane, reflection mirrors 36band 36c, a secondary image forming lens 36d, a diaphragm 36e and a linesensor 36f and the like including a plurality of CCD's.

The focal point detection apparatus 36 in the present embodiment uses awell-known phase difference system. Numeral 37 denotes an imaging platearranged on an anticipated focusing plane of the photographing lens 31,numeral 38 denotes a pentadaha prism for altering a finder optical path,and numerals 39 and 40 denotes an image forming lens and a photometeringsensor, respectively, for measuring an brightness of the object in theview field. The focusing lens 39 is conjugate with the imaging plate 37and the photometering sensor 40 through a reflection optical path in thepentadaha prism 38.

An eyepiece lens 41 having an optical splitter 41a is arranged behind anexit plane of the pentadaha prism 38 and it is used for the observationof the imaging plate 37 by the eye 45 of the photographer. The opticalsplitter 41a comprises a dichroic mirror which transmits a visible rayand reflects an infrared ray.

Numeral 42 denotes a light receiving lens and numeral 44 denotes animage sensor having two-dimensionally arranged a photo-electric elementarray such as CCD's as explained above, which is arranged in conjugateto the vicinity of the pupil of the eye 45 of the photographer which isat a predetermined position with respect to the light receiving lens 42(corresponding to 9 in FIG. 1). Numeral 43 denotes an infrared rayemitting diode which functions as the light source (corresponding to 4in FIG. 1).

Numeral 51 denotes a high intensity superimposing LED which can berecognized even for a bright object. The emitted light is reflected bythe main mirror 32 through the projection lens 52 and verticallydeflected by a fine prism array 37a arranged at a display area of theimaging plate 37 and reaches the eye 45 of the photographer through thepenta prism 38 and the eyepiece lens 41.

The fine prism arrays 37a are formed. in frame shape at a plurality ofpoints (metering points) corresponding to the focus detection area ofthe imaging plate 37, and they are illuminated by five correspondingsuperimposing LED's 51 (which are defined as LED-L1, LED-L2, LED-C,LED-R1 and LED-R2).

Numeral 53 denotes a view field mask which forms a finder view field andnumeral 54 denotes an LCD in the finder for displaying photographinginformation outside of the finder view field. It is illuminated by anillumination LED (F-LED) 55.

The light transmitted through the LCD 54 is introduced into the finderview field by a triangular prism 56 and it is displayed outside of thefinder view field so that the photographer may recognize thephotographing information.

Numeral 61 denotes a diaphragm provided in the photographing lens 31,numeral 64 denotes an aperture driver including an aperture drivecircuit 70 to be described later, numeral 63 denotes a lens drive motor,numeral 64 denotes a lens drive member including drive gears and thelike, and numeral 65 denotes a photo-coupler which detects the rotationof a pulse disk 66 coupled to the lens drive member 64 and transmits itto the lens focal point adjusting circuit 70, which drives the lensdrive motor based on the information from the photo-coupler 65 and thelens driving amount information from the camera to drive thephotographing lens 31 into an in-focus position. Numeral 67 denotes awell-known mount contact point which is an interface to the camera andthe lens.

FIG. 10 shows an electric circuit built in the camera of the presentembodiment, and the like elements to those of FIG. 9 are designated bylike numerals.

Connected to a central processing unit (CPU) 100 of a microcomputerbuilt in the camera body are a visual axis detection circuit 101, aphotometer circuit 102, an automatic focal point detection circuit 103,a signal input circuit 104, an LCD drive circuit 105, an LED drivecircuit 106, an IRED drive circuit 107, a shutter control circuit 108,and a motor control circuit 109. Signals are exchanged with the focusdrive circuit 70 and the aperture drive circuit 111 arranged in thephotographing lens through the mount contact point 67 shown in FIG. 9.

An EEPROM 100a associated with the CPU 100 has a visual axis correctiondata memory function for correcting individual differential errors ofthe visual axis.

As described above, the visual axis detection circuit 101 A/D-convertsthe output of the eyeball image from the image sensor (CCD-EYE) based onthe difference between the output in the illuminated state and theoutput in the non-illuminated state and sends the image information tothe CPU 100, which samples each of the characteristic points of theeyeball image necessary for the detection of the visual axis inaccordance with a predetermined algorithm and calculates the visual axisof the photographer based on the positions of the characteristic points.

The photometer circuit 102 amplifies the output from the photometeringsensor 40, logarithmically compresses it, A/D-converts it, and sends theoutput to the CPU 100 as the luminescence information of each sensor. Inthe present embodiment, the photometering circuit 40 has photo-diodesincluding SPC-L, SPC-C, SPC-R and SPC-A for photometering four areas.

The line sensor 36 of FIG. 10 is a well-known CCD line sensor includingfive line sensors CCD-L2, CCD-L1, CCD-C, CCD-R1 and CCD-R2 correspondingto the five metering points in the image.

The automatic focus detection circuit (focal point detection circuit)103 A/D-converts the voltage obtained from the line sensor 36f and sendsit to the CPU 100. SW-1 denotes a switch which is turned on by a firststroke of a release button to start the photometering, the auto-focusingand the detection of the visual axis, SW-2 denotes a release switchwhich is turned on by a second stroke of the release button, SW-AELdenotes an AE lock switch which is turned on by depressing an AE lockbutton, and SW-DIAL1 and SW-DIAL2 denote dial switches provided in anelectronic dial (not shown) which are connected to an up/down counter ofthe signal input circuit 104 to count on rotation clicks of theelectronic dial.

Numeral 105 denotes a well-known LCD drive circuit for driving theliquid crystal display element LCD. It can display the aperture value,the shutter speed and the preset photographing mode on the monitor LCD72 and the LCD 54 in the finder simultaneously in accordance with thesignal from the CPU 100. The LED drive circuit 106 turns on and off theillumination LED (F-LED) 55 and the superimposing LED 51. The IRED drivecircuit 107 selectively turns on the infrared ray emitting diodes(IRED1-6) according to surrounding states.

The shutter control circuit 108 controls a magnet MG-1, which, whenactuated, drives a leading curtain, and a magnet MG-2 which drives atrailing curtain, to impart a predetermined amount of light exposure toa photosensitive member. The motor control circuit 109 controls a motorM1 which winds up and rewinds a film, and a motor M2 which charges themain mirror 32 and the shutter 34. The shutter control circuit 108 andthe motor control circuit 109 carry out a series of camera releasesequence.

In detecting the visual axis, the eyeball of the subject is illuminatedby the light beam from the illumination means 43, the eyeball image isformed on the accumulation type image pickup means 44 based on thereflected light from the eyeball, the image signal from the image pickupmeans is stored in the memory means 21 (RAM) (FIG. 1), and the visualaxis of the subject is calculated by using the image signal stored inthe memory means. The image pickup means has first and secondaccumulation periods, the memory means stores the image signal of theeyeball image generated in one of the two accumulation periods, and theillumination means emits light in only one of the two accumulationperiods. A difference signal between the image signal from the imagepickup means generated in the first accumulation period and the imagesignal generated in the second accumulation period is determined by thedifferential signal generation means and the line of vision of thesubject is detected by using the signal from the differential signalgeneration means. The high luminescence LED 51 illuminates the pointbased on the calculated visual axis information and the focus isdetected by the focal point detection circuit 103 for the object areacorresponding to the illumination point and the photographing lens 31 isdriven by the focal point adjusting circuit 70.

In accordance with the present invention, when the eyeball image is tobe detected by using the accumulation type image pickup means, theeffect of noise due to external rays is reduced by properly installingthe accumulation method of the eyeball image (image information) by theimage pickup means and the processing method of the image informationbased on the eyeball image from the image pickup means so that thevisual axis detection apparatus which can accurately detect the visualaxis of the eyeball of the photographer (the examined person) who viewsthe finder.

What is claimed is:
 1. A visual axis detection apparatuscomprising:illumination means for illuminating the eye; a singleconversion means, comprising a plurality of pixels arranged over anarea, for converting light from an eye of an examined person a firstelectrical signal when the eye is illuminated by said illumination meansand for converting light from the eye of the examined person to a secondelectrical signal in the absence of illumination of the eye by saidillumination means; signal generation means for generating a signalrepresenting the difference between the first electrical signal and thesecond electrical signal of said single conversion means by obtainingthe difference between outputs of the same pixel during the presence andabsence of illumination of the eye; detection means for detecting avisual axis of the eye on the basis of the signal of said signalgeneration means; and memory means for storing the electrical signalproduced by said conversion means in the state where the eye isilluminated by said illumination means, and wherein said signalgeneration means outputs a signal which is the difference between the anelectrical signal produced by said conversion means in the absence ofillumination and the signal stored in said memory means.
 2. A visualaxis detection apparatus comprising:illumination means for illuminatingthe eye; a single conversion means, comprising a plurality of pixelsarranged over an area, for converting light from an eye of an examinedperson a first electrical signal when the eye is illuminated by saidillumination means and for converting light from the eye of the examinedperson to a second electrical signal in the absence of illumination ofthe eye by said illumination means; signal generation means forgenerating a signal representing the difference between the firstelectrical signal and the second electrical signal of said singleconversion means by obtaining the difference between outputs of the samepixel during the presence and absence of illumination of the eye;detection means for detecting a visual axis of the eye on the basis ofthe signal of said signal generation means; and, wherein said conversionmeans includes a frame transfer-type solid state image pickup meanshaving a first memory and a second memory, wherein said first memorystores the electrical signal of said conversion means in the state wherethe eye is illuminated by said illumination means, and said secondmemory stores the electrical signal of said conversion means in theabsence of illumination.
 3. A visual axis detecting meanscomprising:illumination means for illuminating the eye; photoelectricconversion means provided with a plurality of pixels, each pixel of saidphotoelectric conversion means converting a light beam reflected fromthe eye into an electrical signal; memory means for storing at least oneof the electrical signal of each pixel of said photoelectricalconversion means when said illuminating means illuminates the eye andfor storing the electrical signal of each pixel of said photoelectricalconversion means when said illuminating means does not illuminate theeye; and calculation means for calculating data on a visual axis of theeye based on the difference between each of the electrical signalsproduced by said photoelectrical conversion means when said illuminatingmeans illuminates the eye and each of the electrical signals produced bysaid photoelectrical conversion means when said illuminating means doesnot illuminate the eye, at least one of which has been already stored insaid memory means.